Keyboard with Multi-Symbol Icons

ABSTRACT

A plurality of icons are displayed on a touch-sensitive display. A respective icon in at least a subset of the plurality of icons corresponds to two or more symbols. A first symbol in the two or more symbols belongs to a first subset of symbols and a second symbol in the two or more symbols belongs to a second subset of symbols. The first symbol has a probability of occurrence that is greater than a first pre-determined value and the second symbol has a probability of occurrence that is less than the first pre-determined value. A contact by a user with the touch-sensitive display that corresponds to a selection of the respective icon is detected. The contact includes a respective gesture. A respective symbol in the two or more symbols for the respective icon to which the contact further corresponds is determined.

RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.11/459,606, filed Jul. 24, 2006, entitled “Keyboards for PortableElectronic Devices,” which claims priority to U.S. Provisional PatentApplication No. 60/756,890, filed Jan. 5, 2006, entitled “Keyboards forPortable Electronic Devices,” which applications are incorporated byreference herein in their entirety.

This application is related to U.S. patent application Ser. No.11/459,615, filed Jul. 24, 2006, entitled “Touch Screen Keyboards forPortable Electronic Devices,” which application is incorporated byreference herein in its entirety.

This application is related to U.S. patent application Ser. No.11/228,700, filed Sep. 16, 2005, entitled “Operation of a Computer withTouch Screen Interface,” which application is incorporated by referenceherein in its entirety.

This application is related to U.S. patent application Ser. No.________, filed ________, entitled “Adaptive Keyboard,” Attorney DocketNo. P4150USD1/63266-5270US), which application is incorporated byreference herein in its entirety.

This application is related to U.S. patent application Ser. No.________, filed ________, entitled “User Interface Including WordRecommendations,” Attorney Docket No. P4150USD2/63266-5271US), whichapplication is incorporated by reference herein in its entirety.

This application is related to U.S. patent application Ser. No.________, filed ________, entitled “Keyboard with Multi-Symbol Icons,”Attorney Docket No. P4150USD3/63266-5272US), which application isincorporated by reference herein in its entirety.

TECHNICAL FIELD

The disclosed embodiments relate to user interfaces, and in particular,to user interfaces that include a touch screen keyboard.

BACKGROUND

As portable devices become more compact, and the amount of informationto be processed and stored increases, it has become a significantchallenge to design a user interface that allows users to easilyinteract with the device. This is unfortunate since the user interfaceis the gateway through which users receive not only content but alsoresponses to user actions or behaviors, including user attempts toaccess a device's features or tools. Some portable electronic devices(e.g., mobile phones) have resorted to adding more pushbuttons,increasing a density of push buttons, overloading the functions ofpushbuttons, or using complex menu systems to allow a user to access,store and manipulate data. These conventional user interfaces oftenresult in complicated key sequences and menu hierarchies that must bememorized by the user. In addition, as the number of pushbuttons hasincreased the proximity of neighboring buttons often makes it difficultfor users to activate a desired pushbutton.

Many conventional user interfaces, such as those that include physicalpushbuttons, are also inflexible. This is unfortunate since it mayprevent a user interface from being configured and/or adapted by eitheran application running on the portable device or by users. When coupledwith the time consuming requirement to memorize multiple key sequencesand menu hierarchies, and the difficulty in activating a desiredpushbutton, such inflexibility is frustrating to most users.

Accordingly, there is a need for more transparent and intuitive userinterfaces for portable electronic devices that are easy to use,configure, and/or adapt.

SUMMARY OF EMBODIMENTS

The above deficiencies and other problems associated with userinterfaces for portable devices are reduced or eliminated by thedisclosed touch screen keyboards and their methods of use.

In some embodiments, a method includes displaying a plurality of iconson a touch-sensitive display. A respective icon in at least a subset ofthe plurality of icons corresponds to two or more symbols. A contact bya user with the touch-sensitive display that corresponds to therespective icon is detected. A respective symbol in the two or moresymbols to which the contact further corresponds is determined. Thedisplayed respective icon is modified to indicate that the contactcorresponds to the respective symbol.

The respective symbol may be selected when the user breaks contact withthe respective icon. The respective symbol may be capitalized whencontact is maintained for a time interval exceeding a pre-determinedvalue.

Modifying may include changing a shape of the respective icon. Changingthe shape may include an asymmetric distortion of the shape. An initialshape of the respective icon may include an arc.

Detecting may include detecting rolling of a finger over a region thatcorresponds to the respective symbol. The contact may include a gesturethat is selected from the group consisting of one or more taps, a swipeand a rolling of a finger.

The two or more symbols for the respective icon may be determined inaccordance with a lexicography model. The lexicography model maycorrespond to a user usage history. The user usage history may occurprior to the establishment of the contact. The lexicography model maycorrespond to a frequency of usage of symbols in a language.

In some embodiments, the respective symbol is displayed in a regionwithin the shape of the respective icon and outside of a regioncorresponding to the contact.

In some embodiments, a visual indicator corresponding to the respectivesymbol is provided. The visual indicator may include visual illuminationproximate to the respective icon. The visual illumination may include aband around at least a portion of the respective icon. The visualindicator may be in accordance with a user usage history that occursprior to the detecting of the contact.

In some embodiments, a method includes displaying a plurality of iconson a touch-sensitive display. Two or more subsets of the plurality oficons are arranged in corresponding rows on the touch-sensitive display.A space greater than a pre-determined value is included between adjacentrows. A contact by a user with the touch-sensitive display thatcorresponds to a respective icon is detected. A symbol corresponding tothe respective icon is displayed in the space between a respective rowcorresponding to the respective icon and a neighboring row while thecontact is maintained.

The symbol may be the respective icon. The symbol may be magnifiedrelative to the respective icon. The neighboring row may be above therespective row.

In another embodiment, a plurality of icons are displayed on atouch-sensitive display. A contact by a user with the touch-sensitivedisplay that corresponds to the respective icon is determined. A symbolcorresponding to the respective icon is displayed superimposed over oneor more additional icons in the plurality of icons while the contact ismaintained.

In another embodiment, a plurality of icons are displayed on atouch-sensitive display. Two or more subsets of the plurality of iconsare arranged in corresponding rows. A contact by a user with thetouch-sensitive display that corresponds to the respective icon isdetermined. The displayed plurality of icons are modified to include aspace greater than a pre-determined value between a row corresponding tothe respective icon and an adjacent row on the touch-sensitive displaywhile the contact is maintained. A symbol corresponding to therespective icon is displayed in the space while the contact ismaintained.

In some embodiments, a method includes displaying a plurality of iconson a touch-sensitive display. A respective icon in the plurality oficons corresponds to at least one symbol. One or more recommended wordsare displayed. The one or more recommended words are in accordance witha user history. The one or more recommended words are displayed prior todetecting any contacts by a user corresponding to symbol selection in acurrent application session. A contact by the user with thetouch-sensitive display is detected. The contact includes a gesture. Arespective recommended word corresponding to the gesture is selected.

The gesture may include a swipe motion. The swipe motion may include ahorizontal component with displacement from left to right or from rightto left along the touch-sensitive display. The swipe motion may includea vertical component with displacement downward or upward along thetouch-sensitive display.

The gesture may include one or more taps. A respective tap may includemaking contact with the touch-sensitive display for a time interval lessthan a pre-determined value.

The gesture may include a rolling motion of the contact. The rollingmotion may be from left to right or from right to left along thetouch-sensitive display.

In some embodiments, a method includes displaying a plurality of iconson a touch-sensitive display. A respective icon in at least a subset ofthe plurality of icons corresponds to two or more symbols. A contact bya user with the touch-sensitive display that corresponds to a selectionof the respective icon, wherein the contact includes a respectivegesture, is detected. A respective symbol in the two or more symbols forthe respective icon to which the contact further corresponds isdetermined. The respective symbol is a first symbol in the two or moresymbols if the respective gesture includes a continuous contact and therespective symbol is a second symbol in the two or more symbols if therespective gesture includes a discontinuous contact.

The continuous contact may include a swipe motion. The swipe motion mayinclude a horizontal component with displacement from left to right orfrom right to left along the touch-sensitive display. The swipe motionmay include a vertical component with displacement downward or withdisplacement upward along the touch-sensitive display. The continuouscontact may include a rolling motion of the contact. The rolling motionmay be from left to right or from right to left along thetouch-sensitive display.

The discontinuous contact may include one or more taps. A respective tapmay include contact with the touch-sensitive display for a time intervalless than a first pre-determined value. Two or more consecutive taps maycorrespond to the second symbol if a time interval between two or morecorresponding contacts is less than a second pre-determined value.

The first symbol may be included in a first subset of symbols and thesecond symbol may be included in a second subset of symbols. The firstsubset of symbols may have a probability of occurrence that is greaterthan a first pre-determined value and the second subset of symbols mayhave a probability of occurrence that is less than the firstpre-determined value.

The probability of occurrence may be in accordance with a user history.The probability of occurrence may be in accordance with a lexicographymodel. The lexicography model may include a frequency of usage ofsymbols in a language.

In some embodiments, the second symbol for the respective icon has aprobability of occurrence immediately following the first symbol for therespective icon that is less than a second pre-determined value. In someembodiments, the first symbol for the respective icon has a probabilityof occurrence immediately following the second symbol for the respectiveicon that is less than a second pre-determined value.

In some embodiments, the displayed respective icon is modified toindicate that the contact corresponds to a respective symbol. In someembodiments, a visual indicator corresponding to a respective symbol isprovided. The visual indicator may include visual illumination proximateto the respective icon. The visual illumination may include a bandaround at least a portion of the respective icon.

In some embodiments, a method includes displaying a plurality of iconson a touch-sensitive display. A respective icon in at least a subset ofthe plurality of icons corresponds to two or more symbols. A firstsymbol in the two or more symbols belongs to a first subset of symbolsand a second symbol in the two or more symbols belongs to a secondsubset of symbols. The first symbol has a probability of occurrence thatis greater than a first pre-determined value and the second symbol has aprobability of occurrence that is less than the first pre-determinedvalue. A contact by a user with the touch-sensitive display thatcorresponds to a selection of the respective icon is detected. Thecontact includes a respective gesture. A respective symbol in the two ormore symbols for the respective icon to which the contact furthercorresponds is determined.

The probability of occurrence may be in accordance with a user history.The probability of occurrence may be in accordance with lexicographymodel. The lexicography model may include a frequency of usage ofsymbols in a language. The second symbol may have a probability ofoccurrence immediately following the first symbol that is less than asecond pre-determined value.

The first symbol may be selected using one or more tap gestures and thesecond symbol may be selected using a swipe gesture. A respective tapmay include making contact with the touch-sensitive display for a timeinterval less than a second pre-determined value. Two or moreconsecutive taps may correspond to the second symbol if a time intervalbetween two or more corresponding contacts is less than a thirdpre-determined value.

In some embodiments, the displayed respective icon is modified toindicate that the contact corresponds to the respective symbol. In someembodiments, a visual indicator corresponding to the respective symbolis provided. The visual indicator may include visual illuminationproximate to the respective icon. The visual illumination may include aband around at least a portion of the respective icon.

In some embodiments, the first subset of symbols includes e, t, a, o, i,n, s, r and h. In some embodiments, the first subset of symbols includesq, e, u, I, o, a, d, g, j, l, z, c, b, n and m. In some embodiments, thefirst subset of symbols includes q, c, e, h, I, l, n, o, r, t, u, w andy.

In some embodiments, the second subset of symbols includes w, y and j.In some embodiments, the second subset of symbols includes w, y, p, gand j. In some embodiments, the second subset of symbols includes w, r,t, y, p, s, f, h, k, x and v. In some embodiments, the second subset ofsymbols includes j, v, x and z. In some embodiments, the second subsetof symbols includes b, d, f, g, j, k, m, p, q, s, v, x and z.

The aforementioned methods may be performed by a portable electronicdevice having a touch-sensitive display with a graphical user interface(GUI), one or more processors, memory and one or more modules, programsor sets of instructions stored in the memory for performing thesemethods. In some embodiments, the portable electronic device provides aplurality of functions, including wireless communication.

Instructions for performing the aforementioned methods may be includedin a computer program product configured for execution by one or moreprocessors.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the aforementioned embodiments of theinvention as well as additional embodiments thereof, reference should bemade to the Description of Embodiments below, in conjunction with thefollowing drawings in which like reference numerals refer tocorresponding parts throughout the figures.

FIG. 1 is a block diagram illustrating an embodiment of an architecturefor a portable electronic device.

FIG. 2 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 3A is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 3B is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 3C is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 4 is a flow diagram of an embodiment of a symbol entry process.

FIG. 5 is a block diagram illustrating an embodiment of a character setdata structure.

FIG. 6A is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 6B is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 6C is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 6D is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 7 is a flow diagram of an embodiment of a symbol entry process.

FIG. 8 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 9 is a flow diagram of an embodiment of a symbol entry process.

FIG. 10A is a block diagram illustrating an embodiment of a user wordhistory data structure.

FIG. 10B is a block diagram illustrating an embodiment of a languagedata structure system.

FIG. 11A is a flow diagram of an embodiment of a symbol entry process.

FIG. 11B is a flow diagram of an embodiment of a symbol entry process.

FIG. 11C is a flow diagram of an embodiment of a symbol entry process.

FIG. 12A is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 12B is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 12C is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 12D is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 12E is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 12F is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 12G is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 13 is a flow diagram of an embodiment of a symbol entry process.

FIG. 14 is a flow diagram of an embodiment of a symbol entry process.

FIG. 15 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 16 is a flow diagram of an embodiment of a symbol entry process.

FIG. 17 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

FIG. 18 is a flow diagram of an embodiment of a symbol entry process.

FIG. 19 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device.

DESCRIPTION OF EMBODIMENTS

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the present invention. However, it will beapparent to one of ordinary skill in the art that the present inventionmay be practiced without these specific details. In other instances,well-known methods, procedures, components, and circuits have not beendescribed in detail so as not to unnecessarily obscure aspects of theembodiments.

Embodiments of user interfaces and associated processes for using adevice are described. In some embodiments, the device may be a portablecommunications device. The user interface may include a click wheeland/or touch screen. A click wheel is a physical user-interface devicethat may provide navigation commands based on an angular displacement ofthe wheel or a point of contact with the wheel by a user of the device.A click wheel may also be used to provide a user command correspondingto selection of one or more items, for example, when the user of thedevice presses down on at least a portion of the wheel. For simplicity,in the discussion that follows, a portable communications device (e.g.,a cellular telephone that may also contain other functions, such as SMS,PDA and/or music player functions) that includes a touch screen is usedas an exemplary embodiment. It should be understood, however, that theuser interfaces and associated processes may be applied to otherdevices, such as personal computers and laptops, that may include one ormore other physical user-interface devices, such as a click wheel, akeyboard, a mouse and/or a joystick.

The device may support a variety of applications, such as a telephone,text messaging, word processing, email and a music player. The musicplayer may be compatible with one or more file formats, such as MP3and/or AAC. In an exemplary embodiment, the device includes an iPodmusic player (trademark of Apple Computer, Inc.).

The various applications that may be executed on the device may use atleast one common physical user-interface device, such as the touchscreen. In embodiments that include a click wheel, one or more functionsof the click wheel as well as corresponding information displayed on thedevice may be adjusted and/or varied from one application to the nextand/or within a respective application. In this way, a common physicalarchitecture (such as the click wheel) of the device may support thevariety of applications with user interfaces that are intuitive andtransparent.

The user interfaces may include one or more keyboard embodiments. Thekeyboard embodiments may include standard (qwerty) and/or non-standardconfigurations of symbols on the displayed icons of the keyboard. Thekeyboard embodiments may include a reduced number of icons (or softkeys) relative to the number of keys in existing physical keyboards,such as that for a typewriter. This may make it easier for users toselect one or more icons in the keyboard, and thus, one or morecorresponding symbols. The keyboard embodiments may be adaptive. Forexample, displayed icons may be modified in accordance with useractions, such as selecting one or more icons and/or one or morecorresponding symbols. One or more applications on the portable devicemay utilize common and/or different keyboard embodiments. Thus, thekeyboard embodiment used may be tailored to at least some of theapplications. In some embodiments, one or more keyboard embodiments maybe tailored to a respective user. For example, based on a word usagehistory (lexicography, slang, individual usage) of the respective user.Some of the keyboard embodiments may be adjusted to reduce a probabilityof a user error when selecting one or more icons, and thus one or moresymbols, when using the keyboard embodiments.

Attention is now directed towards embodiments of the device. FIG. 1 is ablock diagram illustrating an architecture for a portable electronicdevice 100, according to some embodiments of the invention. The device100 may include a memory 102 (which may include one or more computerreadable storage mediums), a memory controller 122, one or moreprocessing units (CPU's) 120, a peripherals interface 118, RF circuitry108, audio circuitry 110, a speaker 111, a microphone 113, aninput/output (I/O) subsystem 106, a display system 112 (which mayinclude a touch screen), a click wheel 114, other input or controldevices 116, and an external port 124. These components may communicateover the one or more communication buses or signal lines 103. The device100 may be any portable electronic device, including but not limited toa handheld computer, a tablet computer, a mobile phone, a media player,a personal digital assistant (PDA), or the like, including a combinationof two or more of these items. In other embodiments, the device 100 maynot be portable, such as a personal computer.

It should be appreciated that the device 100 is only one example of aportable electronic device 100, and that the device 100 may have more orfewer components than shown, may combine two or more components, or amay have a different configuration or arrangement of the components. Thevarious components shown in FIG. 1 may be implemented in hardware,software or a combination of both hardware and software, including oneor more signal processing and/or application specific integratedcircuits.

The memory 102 may include high speed random access memory and may alsoinclude non-volatile memory, such as one or more magnetic disk storagedevices, flash memory devices, or other non-volatile solid state memorydevices. In some embodiments, the memory 102 may further include storageremotely located from the one or more processors 120, for instancenetwork attached storage accessed via the RF circuitry 108 or theexternal port 124 and a communications network (not shown) such as theInternet, intranet(s), Local Area Networks (LANs), Wide Local AreaNetworks (WLANs), Storage Area Networks (SANs) and the like, or anysuitable combination thereof. Access to the memory 102 by othercomponents of the device 100, such as the CPU 120 and the peripheralsinterface 118, may be controlled by the memory controller 122.

The peripherals interface 118 couples the input and output peripheralsof the device to the CPU 120 and the memory 102. The one or moreprocessors 120 run or execute various software programs and/or sets ofinstructions stored in the memory 102 to perform various functions forthe device 100 and to process data.

In some embodiments, the peripherals interface 118, the CPU 120, and thememory controller 122 may be implemented on a single chip, such as achip 104. In some other embodiments, they may be implemented on separatechips.

The RF (radio frequency) circuitry 108 receives and sendselectromagnetic waves. The RF circuitry 108 converts electrical signalsto/from electromagnetic waves and communicates with communicationsnetworks and other communications devices via the electromagnetic waves.The RF circuitry 108 may include well-known circuitry for performingthese functions, including but not limited to an antenna system, an RFtransceiver, one or more amplifiers, a tuner, one or more oscillators, adigital signal processor, a CODEC chipset, a subscriber identity module(SIM) card, memory, and so forth. The RF circuitry 108 may communicatewith the networks, such as the Internet, also referred to as the WorldWide Web (WWW), an Intranet and/or a wireless network, such as acellular telephone network, a wireless local area network (LAN) and/or ametropolitan area network (MAN), and other devices by wirelesscommunication. The wireless communication may use any of a plurality ofcommunications standards, protocols and technologies, including but notlimited to Global System for Mobile Communications (GSM), Enhanced DataGSM Environment (EDGE), wideband code division multiple access (W-CDMA),code division multiple access (CDMA), time division multiple access(TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over Internet Protocol(VoIP), Wi-MAX, a protocol for email, instant messaging, and/or ShortMessage Service (SMS)), or any other suitable communication protocol,including communication protocols not yet developed as of the filingdate of this document.

The audio circuitry 110, the speaker 111, and the microphone 113 providean audio interface between a user and the device 100. The audiocircuitry 110 receives audio data from the peripherals interface 118,converts the audio data to an electrical signal, and transmits theelectrical signal to the speaker 111. The speaker 111 converts theelectrical signal to human-audible sound waves. The audio circuitry 110also receives electrical signals converted by the microphone 113 fromsound waves. The audio circuitry 110 converts the electrical signal toaudio data and transmits the audio data to the peripherals interface 118for processing. Audio data may be may be retrieved from and/ortransmitted to the memory 102 and/or the RF circuitry 108 by theperipherals interface 118. In some embodiments, the audio circuitry 110also includes a headset jack (not shown). The headset jack provides aninterface between the audio circuitry 110 and removable audioinput/output peripherals, such as output-only headphones or a headsetwith both output (headphone for one or both ears) and input(microphone).

The I/O subsystem 106 provides the interface between input/outputperipherals on the device 100, such as the display system 112, the clickwheel 114 and other input/control devices 116, and the peripheralsinterface 118. The I/O subsystem 106 may include a display controller156, a click wheel controller 158 and one or more input controllers 160for other input or control devices. The one or more input controllers160 receive/send electrical signals from/to other input or controldevices 160. The other input/control devices 160 may include physicalbuttons (e.g., push buttons, rocker buttons, etc.), dials, sliderswitches, sticks, and so forth.

The display system 112 provides an output interface and/or an inputinterface between the device and a user. The display controller 156receives and/or sends electrical signals from/to the display system 112.The display system 112 displays visual output to the user. The visualoutput may include text, icons, graphics, video, and any combinationthereof. In some embodiments, some or all of the visual output maycorrespond to user-interface objects, further details of which aredescribed below.

In some embodiments, such as those that include a touch screen, thedisplay system 112 also accepts input from the user based on hapticand/or tactile contact. In embodiments with a touch screen, the displaysystem 112 forms a touch-sensitive surface that accepts user input. Inthese embodiments, the display system 112 and the display controller 156(along with any associated modules and/or sets of instructions in thememory 102) detect contact (and any movement or breaking of the contact)on the display system 112 and converts the detected contact intointeraction with user-interface objects, such as one or more soft keys,that are displayed on a touch screen. In an exemplary embodiment, apoint of contact between a touch screen in the display system 112 andthe user corresponds to one or more digits of the user.

In embodiments with a touch screen, the touch screen in the displaysystem 112 may use LCD (liquid crystal display) technology, or LPD(light emitting polymer display) technology, although other displaytechnologies may be used in other embodiments. A touch screen in thedisplay system 112 and the display controller 156 may detect contact andany movement or breaking thereof using any of a plurality of touchsensitivity technologies, including but not limited to capacitive,resistive, infrared, and surface acoustic wave technologies, as well asother proximity sensor arrays or other elements for determining one ormore points of contact with a touch screen in the display system 112. Atouch-sensitive display in some embodiments of the display system 112may be analogous to the multi-touch sensitive tablets described in thefollowing U.S. Pat. No. 6,323,846 (Westerman et al.), U.S. Pat. No.6,570,557 (Westerman et al.), and/or U.S. Pat. No. 6,677,932(Westerman), and/or U.S. Patent Publication 2002/0015024A1, each ofwhich is hereby incorporated by reference. However, a touch screen inthe display system 112 displays visual output from the portable device100, whereas touch sensitive tablets do not provide visual output. Thetouch screen in the display system 112 may have a resolution in excessof 100 dpi. In an exemplary embodiment, the touch screen in the displaysystem may have a resolution of approximately 168 dpi. The user may makecontact with the touch screen in the display system 112 using anysuitable object or appendage, such as a stylus, finger, and so forth.

In some embodiments, in addition to touch screen, the device 100 mayinclude a touchpad (not shown) for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad may be a touch-sensitive surface that is separatefrom the touch screen in the display system 112 or an extension of thetouch-sensitive surface formed by the touch screen.

The device 100 may include a click wheel 114. A user may navigate amongone or more graphical objects (henceforth referred to as icons)displayed in the display system 112 by rotating the click wheel 114 orby moving (e.g., angular displacement) of a point of contact with theclick wheel 114. The click wheel 114 may also be used to select one ormore of the displayed icons. For example, the user may press down on atleast a portion of the click wheel 114 or an associated physical button.User commands and navigation commands provided by the user via the clickwheel 114 may be processed by the click wheel controller 158 as well asone or more of the modules and/or sets of instructions in the memory102.

The device 100 also includes a power system 162 for powering the variouscomponents. The power system 162 may include a power management system,one or more power sources (e.g., battery, alternating current (AC)), arecharging system, a power failure detection circuit, a power converteror inverter, a power status indicator (e.g., a light-emitting diode(LED)) and any other components associated with the generation,management and distribution of power in portable devices.

In some embodiments, the software components stored in the memory 102may include an operating system 126, a communication module (or set ofinstructions) 128, a contact/motion module (or set of instructions) 130,a graphics module (or set of instructions) 132, one or more applications(or set of instructions) 136, a timer module (or set of instructions)144, a word prediction module (or set of instructions) 146, an addressbook 148, a user word history 150, one or more character sets 152, andone or more lexicography models 154. The graphics module 132 may includean icon effects module (or set of instructions) 134. The applicationsmodule 136 may include a telephone module (or set of instructions) 138,a text messaging module (or set of instructions) 140 and/or a musicplayer module (or set of instructions) 142.

The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X,WINDOWS, or an embedded operating system such as VxWorks) includesvarious software components and/or drivers for controlling and managinggeneral system tasks (e.g., memory management, storage device control,power management, etc.) and facilitates communication between varioushardware and software components.

The communication module 128 facilitates communication with otherdevices over one or more external ports 124 and also includes varioussoftware components for handling data received by the RF circuitry 108and/or the external port 124. The external port 124 (e.g., UniversalSerial Bus (USB), FIREWIRE, etc.) is adapted for coupling directly toother devices or indirectly over a network (e.g., the Internet, wirelessLAN, etc.).

The contact/motion module 130 may detect contact with the click wheel114 and/or a touch screen in the display system 112 (in conjunction withthe display controller 156). The contact/motion module 130 includesvarious software components for performing various operations related todetection of contact, such as determining if contact has occurred,determining if there is movement of the contact and tracking themovement across the click wheel 114 and/or a touch screen in the displaysystem 112, and determining if the contact has been broken (i.e., if thecontact has ceased). Determining movement of the point of contact mayinclude determining speed (magnitude), velocity (magnitude anddirection), and/or an acceleration (including magnitude and/ordirection) of the point of contact. In some embodiments, thecontact/motion module 130 and the display controller 156 also detectcontact on a touchpad.

The graphics module 132 includes various known software components forrendering and displaying graphics on the display system 112. Note thatthe term “graphics” includes any object that can be displayed to a user,including without limitation text, web pages, icons (such asuser-interface objects including soft keys), digital images, videos,animations and the like.

In some embodiments, the graphics module 132 includes the icon effectsmodule 134. The icon effects module 134 may modify a displayed positionof one or more icons on the display system 112 (in conjunction with thedisplay controller 156) based on user actions (such as detecting acontact corresponding to at least one icon). In some embodiments, themodification of the displayed icon(s) may be based on an animationsequence.

In addition to the telephone module 138, the text messaging module 140and/or the music player module 142, the one or more applications 136 mayinclude any applications installed on the device 100, including withoutlimitation, a browser, the address book 148, contact list, email,instant messaging, word processing, keyboard emulation, widgets,JAVA-enabled applications, encryption, digital rights management, voicerecognition, voice replication, location determination capability (suchas that provided by the global positioning system (GPS)), etc.

In conjunction with the RF circuitry 108, the audio circuitry 110, thespeaker 111, the microphone 113, the display system 112, the displaycontroller 156, the click wheel 114 and/or the click wheel controller158, the telephone module 138 may be used to enter a sequence ofcharacters corresponding to a telephone number, access one or moretelephone numbers in the address book 148, modify a telephone numberthat has been entered, dial a respective telephone number, conduct aconversation and disconnect or hang up when the conversation iscompleted.

In conjunction with the display system 112, the display controller 156,the click wheel 114 and/or the click wheel controller 158, the textmessaging module 140 may be used to enter a sequence of characterscorresponding to a text message, to modify previously enteredcharacters, to transmit a respective text message (for example, using aShort Message Service or SMS protocol), to receive text messages and toview received text messages. In some embodiments, transmitted and/orreceived text messages may include graphics, photos, audio files, videofiles and/or other attachments as are supported in a Multimedia MessageService (MMS) and/or an Enhanced Messaging Service (EMS). Embodiments ofuser interfaces and associated processes corresponding to the symbolentry, such as with the text messaging module 140, and more generally,to text entry and communication are described further below withreference to FIGS. 2-4, 6-9 and 11-20.

In conjunction with the display system 112, the display systemcontroller 156, the click wheel 114, the click wheel controller 158, theaudio circuitry 110, the speaker 111 and/or the microphone 113, themusic player module 142 allows the user to play back recorded musicstored in one or more files, such as MP3 or AAC files. In someembodiments, the device 100 may include the functionality of an MP3player, such as an iPod (trademark of Apple Computer, Inc.). The device100 may, therefore, include a 36-pin connector that is compatible withthe iPod.

The timer module 144 may provide a time reference and/or time stamps foruser commands received by the device 100, for example, using the clickwheel 114 and the click wheel controller 158.

The word prediction module 146 may be used in conjunction with one ormore of the applications 136, such as the text messaging module 140. Theword prediction module 146 may suggest one or more words or symbols(such as punctuation marks, pronunciation marks or spaces) in accordancewith a context. The context may be based on one or more of thelexicography models 154 (for example, grammatical and/or syntax rulesassociated with one or more languages) and/or a user word history 150.The context may include one or more previously entered words,characters, and/or symbols. The context may depend on which of theapplications 136 is being used. For example, there may be differentcontexts for an email application as opposed to a word processingapplication. A user interface and associated process that includerecommended words from the word prediction module 146 are discussedfurther below with reference to FIGS. 8 and 9.

The user word history 150 may include static content (such as thatassociated with a dictionary) and/or dynamic content (such as thatassociated with characters, symbols and/or words that are routinelyand/or recently used by the user). The user word history 150 may includea static dictionary built up by scanning a user's address book, emails,and other documents. The user word history 150 may include weightedscores or probabilities for predicted words based on a set ofcharacters, symbols and/or words that are provided by the user to thedevice 100, for example, using the display system 112, the click wheel114 and the click wheel controller 158. The user word history 150 mayalso include use statistics (e.g., time of use and/or frequency of use)of one or more characters, symbols and/or words that are provided by theuser. The user word history 150 is discussed further below withreference to FIGS. 10A and 10B.

The character sets 152 may include one or more sets of characterscorresponding to numbers, letters and/or symbols. The letters and/orsymbols may correspond to one or more languages. The character sets 152may be used by one or more of the applications 136, such as the textmessaging module 140. A data structure associated with the one or morecharacter sets (which may be used in one or more of the keyboardembodiments) is discussed further below with reference to FIG. 5.

In some embodiments, the device 100 may include one or more optionaloptical sensors (not shown), such as CMOS or CCD image sensors, for usein imaging applications.

In some embodiments, the device 100 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen in the display system 112 and/or a touchpad. Byusing a touch screen and/or a touchpad as the primary input/controldevice for operation of the device 100, the number of physicalinput/control devices (such as push buttons, dials, and the like) on thedevice 100 may be reduced. In one embodiment, the device 100 includes atouch screen, a touchpad, a push button for powering the device on/offand locking the device, a volume adjustment rocker button and a sliderswitch for toggling ringer profiles. The push button may be used to turnthe power on/off on the device by depressing the button and holding thebutton in the depressed state for a predefined time interval, or may beused to lock the device by depressing the button and releasing thebutton before the predefined time interval has elapsed. In analternative embodiment, the device 100 also may accept verbal input foractivation or deactivation of some functions through the microphone 113.

The predefined set of functions that are performed exclusively through atouch screen and/or a touchpad include navigation between userinterfaces. In some embodiments, the touchpad, when touched by the user,navigates the device 100 to a main, home, or root menu from any userinterface that may be displayed on the device 100. In such embodiments,the touchpad may be referred to as a “menu button.” In some otherembodiments, the menu button may be a physical push button or otherphysical input/control device instead of a touchpad.

In some embodiments, the device 100 is a device where operation of apredefined set of functions on the device is performed exclusively orprimarily through the click wheel 114. By using the click wheel 114 asthe primary input/control device for operation of the device 100, thenumber of other physical input/control devices (such as push buttons,dials, and the like) on the device 100 may be reduced.

Attention is now directed towards embodiments of user interfaces andassociated processes that may be implemented on the device 100. FIG. 2is a schematic diagram illustrating an embodiment of a user interfacefor a portable electronic device 200. The device 200 includes a touchscreen 208. The touch screen may display one or more trays. A tray is aregion within a graphical user interface. One tray may include a userentry interface, such as a keyboard 210 that includes a plurality oficons. The icons may include one or more symbols. In this embodiment, aswell as others described below, a user may select one or more of theicons, and thus, one or more of the corresponding symbols, by makingcontact or touching the keyboard 210, for example, with one or morefingers 212 (not drawn to scale in the figure). The contact maycorrespond to the one or more icons. In some embodiments, selection ofone or more icons occurs when the user breaks contact with the one ormore icons. In some embodiments, the contact may include a gesture, suchas one or more taps, one or more swipes (from left to right, right toleft, upward and/or downward) and/or a rolling of a finger (from rightto left, left to right, upward and/or downward) that has made contactwith the device 200. In some embodiments, in advertent contact with anicon may not select a corresponding symbol. For example, a swipe gesturewith an icon may not select a corresponding symbol when the gesturecorresponding to selection is a tap.

The device 200 may include a display tray 214. The display tray 214 maydisplay one or more of the characters and/or symbols that are selectedby the user. The device 200 may also include one or more physicalbuttons, such as the clear, hold and menu buttons shown in FIG. 2. Asdescribed previously, the menu button may be used to navigate within ahierarchy of applications that may be executed on the device 200.Alternatively, in some embodiments, the clear, hold, and/or menu buttonsare implemented as soft keys in a GUI in touch screen 208.

FIGS. 3A-3C are schematic diagrams illustrating an embodiment of a userinterface for a portable electronic device 300. The user interfaceincludes a keyboard 310 that includes a plurality of icons. The iconsinclude three symbols each. In other embodiments, the icons include twosymbols each. In other embodiments, different icons on the same keyboardmay include one, two, or three symbols each (e.g., some icons maycontain one symbol while other icons contain two or three symbols). Thesymbols on the icons are in a non-standard configuration, i.e.,non-qwerty. In addition, the total number of icons in the keyboard 310is less than the number of physical keys in a standard keyboard.

The symbols in the icons in the keyboard 310 may be determined using alexicography model, such as a language. The lexicography model mayinclude a frequency of use of symbols in a language. For example,characters or symbols that are unlikely to occur immediately proximateto one another or immediately after one another in a set of symbols thatthe user may enter may be grouped on a respective icon 312 (FIG. 3B). Alanguage may include slang as well as individual usage (for example,words that are commonly used by the user). The lexicography model maycorrespond to a user usage or word history that occurs prior to the usermaking contact with the device 300, i.e., a past usage.

As shown in FIG. 3B, when a user makes contact 314 with the touch screen208 in the device 300 corresponding to the respective icon 312 and arespective symbol (in this case a letter ‘a’), the shape of therespective icon 312 is modified. This provides information to the useras to which icon and which symbol the contact 314 currently corresponds.This may be useful since the contact 314 may obscure at least a portionof the respective icon 312 making it difficult for the user to see therespective symbol he or she is currently positioned on.

In an exemplary embodiment, the icons in the keyboard 310 may at leastin part include an arc. In response to the contact 314, the shape of therespective icon 312 may be asymmetrically distorted and the respectivesymbol that the contact 314 currently corresponds to may be displayedwithin the shape of the respective icon 312 and outside of the contact314.

In some embodiments, the user may select the respective symbol by makingthe contact 314 with the respective icon 312 and rolling a finger over aregion within the respective icon 312 that corresponds to the respectivesymbol. If the user determines, based on the modified shape of therespective icon 312 and/or the displayed symbol within the modifiedshape that the wrong symbol is currently contacted, the user may rolltheir finger to a different position within the respective icon 312 thatcorresponds to the correct symbol. Once the contact 314 has beenpositioned over or proximate to the correct symbol, the user may selectthis symbol by breaking the contact 314 with the respective icon 312.The selected symbol (such as the letter ‘a’) may then be displayed inthe display tray 214. In some embodiments, if the contact 314 ismaintained by the user for a time interval that is more than a firstpre-determined value, such as 0.5, 1 or 2 s, before the contact 314 isbroken, the respective symbol may be capitalized.

If an error has been made, the user may clear the entire display tray214 using a clear icon or may delete a most recently selected symbolusing a delete icon. Once a set of symbols (such as a message) has beenentered, the user may accept the set of symbols (which may store and/orsend the set of symbols depending on the application executing on thedevice 300) using an accept icon.

As shown in FIG. 3C, in some embodiments an additional visual indicatorcorresponding to the respective icon 312 may be provided on the display208. The visual indicator may be proximate to the respective icon 312.The visual indicator may include a band 318 around at least a portion ofthe respective icon 312.

As is also shown in FIG. 3C, in some embodiments a shape of therespective icon 312 may not be modified in response to the contact 314.Instead, an icon 316 corresponding to the respective symbol 316 may bedisplayed proximate to the respective icon 312.

The modifying of the shape of the respective icon 312 and/or thedisplaying of the visual indicator, such as the band 318 and/or the icon316, may be included in at least some of the embodiments discussedfurther below.

While the device 300 has been illustrated with certain components and aparticular arrangement of these components, it should be understood thatthere may be fewer or more components, two or more components may becombined, and positions of one or more components may be changed. Forexample, the keyboard 310 may include fewer or additional icons. In someembodiments, a different character set and/or different groups ofsymbols may be used on the icons in the keyboard 310.

FIG. 4 is a flow diagram of an embodiment of a symbol entry process 400.While the symbol entry process 400 described below includes a number ofoperations that appear to occur in a specific order, it should beapparent that the process 400 can include more or fewer operations,which can be executed serially or in parallel (e.g., using parallelprocessors or a multi-threading environment), an order of two or moreoperations may be changed and/or two or more operations may be combinedinto a single operation.

A plurality of icons may be displayed on a touch-sensitive display(410). A respective icon may correspond to two or more symbols. Contactby a user with the display that corresponds to the respective icon maybe detected (412). The displayed respective icon may be modified toindicate that the contact corresponds to a respective symbol in the twoor more symbols (414). The respective symbol may be optionally displayedin a region within the shape of the respective icon and outside of aregion corresponding to the contact (416). A visual indicatorcorresponding to the respective symbol may be optionally provided (418).The respective symbol may be optionally capitalized when contact ismaintained for a time interval exceeding a pre-determined value (420).The respective symbol may be selected when the user breaks contact withthe respective icon (422).

Attention is now directed towards embodiments of a character set datastructure that may be used in implementing the user interface in thedevice 300 (FIG. 3) and/or user interfaces described further below. FIG.5 is a block diagram illustrating an embodiment of a character set datastructure 500. The character sets 152 may include multiple sets 512 ofcharacters and/or symbols. A respective set, such as the set 512-1, mayinclude one or more symbols 514 and one or more probabilities 516. Theprobabilities may include frequencies of occurrence of use, as well asconditional probabilities (such as the probability of a given symboloccurring given one or more symbols that have already occurred). In someembodiments the character set data structure 500 may include fewer ormore components. Two or more components may be combined and an order oftwo or more components may be changed.

Attention is now directed towards additional embodiments of userinterfaces and associated processes that may be implemented on thedevice 100 (FIG. 1). FIGS. 6A-6D are schematic diagrams illustrating anembodiment of a user interface for a portable electronic device 600. Thedevice 600 includes a keyboard 610 that has a plurality of iconsarranged in rows. A given row includes a subset of the plurality oficons. Adjacent rows are separated by a space greater than a secondpre-determined value, such as a height of one of the icons.

As shown in FIG. 6B, when the user makes a contact 612 with the display208 corresponding to a respective icon in the keyboard 610, an icon 614may be displayed in the space between two adjacent rows. The icon maycorrespond to a respective symbol that corresponds to the respectiveicon that the user has contacted 612. For example, if the user contactsor is proximate to an icon for the character ‘u’ in the keyboard 610,the icon 614 may correspond to the character ‘u’. In this way, the usermay receive feedback that the respective icon (and thus, the respectivesymbol) is currently contacted. This may be useful because the contact612 may obscure the respective icon, and thus, the respective symbol,that has been selected in the rows of icons.

In some embodiments, the icon 614 may be displayed above a respectiverow in which the contact 612 has occurred. In some embodiments, the icon614 may be magnified, i.e., larger, than the respective icon.

The icon 614 may be displayed while the contact 612 is maintained. Whenthe user breaks the contact 612 with the respective icon, the respectivesymbol may be selected. In some embodiments, the respective symbol maybe displayed in the display tray 214.

As shown in FIG. 6C, in some embodiments a keyboard 616 may be displayedwith rows of icons. Initially, the rows of icons may not include asignificant space between adjacent rows, e.g., the space may be lessthan the second pre-determined value. When the user makes the contact612 with the display 208, however, the displayed keyboard 616 may bemodified to include a space greater the second pre-determined value andthe icon 614 may be displayed. This modified configuration or layout ofthe keyboard 616 may be maintained while the contact 612 is maintainedby the user.

As shown in FIG. 6D, in some embodiments a keyboard 618 may include rowsof icons. When the contact 612 is made, an icon 620 may be displayedsuperimposed over at least one or more additional icons in the keyboard618.

While the device 600 has been illustrated with certain components and aparticular arrangement of these components, it should be understood thatthere may be fewer or more components, two or more components may becombined, and positions of one or more components may be changed. Forexample, the keyboards 610, 616 and/or 618 may include fewer oradditional icons. In some embodiments, a different character set and/ordifferent groups of symbols may be used on the icons in the keyboards610, 616 and/or 618.

FIG. 7 is a flow diagram of an embodiment of a symbol entry process 700.While the symbol entry process 700 described below includes a number ofoperations that appear to occur in a specific order, it should beapparent that the process 700 can include more or fewer operations,which can be executed serially or in parallel (e.g., using parallelprocessors or a multi-threading environment), an order of two or moreoperations may be changed and/or two or more operations may be combinedinto a single operation.

A plurality of icons may be displayed on a touch-sensitive display(710). Two or more subsets of the plurality of icons may be arranged inrows. A contact by a user with the display that corresponds to arespective icon may be detected (712). A symbol corresponding to therespective icon may be optionally displayed between a row correspondingto the respective icon and a neighboring row (714). A symbolcorresponding to the respective icon may be optionally displayedsuperimposed over one or more additional icons in the plurality of icons(716).

FIG. 8 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device 800. The device 800 mayinclude a tray 812 that includes one or more recommended words 810. Theone or more recommended words 810 may be determined using a user wordhistory. This is discussed further below with reference to FIGS. 10A and10B.

In some embodiments, the one or more recommended words 810 are displayedprior to detecting any contacts corresponding to text input (symbolselection) by the user in a current application session. For example,the one or more recommended words 810 may be displayed when the userinitially opens an application, such as email, on the device 800. Theone or more recommended words 810, therefore, may be determined based ona user word or usage history that may be application specific. After thedevice 800 receives contacts corresponding to text input, the one ormore recommended words 810 may change dynamically in response tocontacts corresponding to text input by the user during the applicationsession.

The user may select one or more of the recommended words 810 by makingcontact with the display 208. In some embodiments, one or more of therecommended words 810, such as a phrase (“How are you?”), may beselected with a single contact. The contact may include a gesture, suchas one or more taps, one or more swipes, and/or a rolling motion of afinger that makes the contact. The one or more taps may have a durationthat is less than a third pre-determined value, such as 0.1, 0.5 or 1 s.

While the device 800 has been illustrated with certain components and aparticular arrangement of these components, it should be understood thatthere may be fewer or more components, two or more components may becombined, and positions of one or more components may be changed. Forexample, the keyboard 210 may include fewer or additional icons. In someembodiments, a different character set and/or different groups ofsymbols may be used on the icons in the keyboard 210.

FIG. 9 is a flow diagram of an embodiment of a symbol entry process 900.While the symbol entry process 900 described below includes a number ofoperations that appear to occur in a specific order, it should beapparent that the process 900 can include more or fewer operations,which can be executed serially or in parallel (e.g., using parallelprocessors or a multi-threading environment), an order of two or moreoperations may be changed and/or two or more operations may be combinedinto a single operation.

A plurality of icons may be displayed on a touch-sensitive display(910). A respective icon may correspond to at least one symbol. One ormore recommended words may be displayed (912). The one or morerecommended words may be in accordance with a user history prior todetecting any contacts corresponding to text input (symbol selection) bythe user in a current application session. A contact by the user withthe display may be detected (914). The contact may include a gesture. Arespective recommended word that corresponds to the gesture may beselected (916).

Attention is now directed towards embodiments of data structure systemsthat may be implementing in the device 100 (FIG. 1). FIG. 10A is a blockdiagram illustrating an embodiment of a user word history data structure1000. The user word history 150 may include a deleted word stack 1010and multiple words 1016. The words 1016 may include one or morecharacters and/or one or more symbols. The deleted word stack 1010includes one or more words 1014 in a sequential order in which the oneor more words 1014 were deleted by the user in an application, such asthe text messaging module 140 (FIG. 1).

A respective word in the words 1016, such as word 1016-M, may includemultiple records. A respective record may include a time-weighted score1018, use statistics 1020 (such as a time of use and/or a frequency ofuse), a context 1022 and one or more applications 1024. Thetime-weighted score 1018 may indicate a probability that the word 1016-Mis a next predicted word based on the context 1022 (one or morecharacters, symbols and/or words that have previously been provided bythe user) and/or the application 1024. For example, the time-weightedscore 1018 may therefore be different for email than for the textmessaging module 140 (FIG. 1). The time-weighted score 1018 may becomputed to favorably weight (e.g., give a higher probability) to wordsthat are used recently. For example, the time-weighted score 1018 maygive favorable weighting to words 1016 that are used within the last 24hours or week. Words 1016 used on longer time scales (e.g., more than aday or a week ago) may have their corresponding time-weighted scores1018 reduced by a pre-determined ratio (such as 0.9) for each additionaltime interval (e.g., each day or week) since the words 1016 were lastused.

The user history data structure 1000 may include static information (forexample, corresponding to a dictionary and/or grammatical and syntaxrules for one or more languages) as well as dynamic information (basedon recent usage statistics and/or patterns). Thus, the user history datastructure 1000 may be dynamically updated continuously, afterpre-determined time intervals, or when a new word or syntax is employedby the user. The user history data structure 1000 may include a staticdictionary built up by scanning a user's address book, emails, and otherdocuments. In some embodiments the user history data structure 1000 mayinclude fewer or more components. Two or more components may be combinedand an order of two or more components may be changed.

FIG. 10B is a block diagram illustrating an embodiment of a languagedata structure system 1050. The language data structure system 1050 maybe used to provide recommended words in the device 800 (FIG. 8). Asequence of symbols 1062 (including one or more characters, symbolsand/or words) may be provided by the user. A set of symbols 1062corresponding to a context 1022-1 may be processed by a context map1060. In some embodiments, the context 1022-1 may be a null set, i.e.,one or more recommended words are provided before the user provides anysymbols 1062 (e.g., when an application is first opened). In otherembodiments, the context 1022-1 may include one or more previouslyentered or provided words as well as one or more symbols, such as thefirst one, two or three letters in a current word that the user isproviding. The context map 1060 may include a select and hashing module1064 and a hash map 1066. The hash map 1066 may select one or moreappropriate entries in an application-specific dictionary 1068. Theentries in the application-specific dictionary 1068 may include contexts1070, predicted words 1072, and time-weighted scores 1074. Theapplication-specific dictionary 1068 may utilize the records in the userhistory data structure 1000. As a consequence, the application-specificdictionary 1068 may be dynamically updated continuously, afterpre-determined time intervals, or when a new word or syntax is employedby the user.

The language data structure system 1050 may be used to provide one ormore recommended words based on the context 1022-1. The context map mayfind a top-5 or top-10 best context 1070 matches. The correspondingpredicted words 1072 may be recommended to the user in accordance withthe time-weighted scores 1074. In some embodiments, only a subset of thepredicted words 1072 corresponding to the best context 1070 matches maybe presented to the user (e.g., just the top-1, top-2, or top-3predicted words).

In some embodiments, the language data structure system 1050 may provideone or more recommended words in accordance with a state machine(corresponding to a Markov sequence or process) that corresponds to alanguage. For example, the application-specific dictionary 1068 may bebased on a stochastic model of the relationships among letters,characters, symbols and/or words in a language.

A path memory (such as up to three characters in a word that iscurrently being entered and/or two or three previously entered words) ofthe probabilistic model represents a tradeoff between accuracy and theprocessing and power capabilities (for example, battery life) of theportable electronic device 100 (FIG. 1). In some embodiments, such aprobabilistic model may be based on a lexicography and usage that isuser-specific and/or, as discussed previously, even applicationspecific. For example, user emails, address book and/or other documentsmay be analyzed to determine an appropriate probabilistic model for thatuser based on the syntax and/or lexicography (including names and slang)that are employed by the user. The probabilistic model may be updatedcontinuously, after pre-determined time intervals, or when a new word orsyntax is employed by the user.

In some embodiments, the probabilistic model may be based on one or moremistakes made by the user when using the click wheel 114 (FIG. 1) and/ora touch-sensitive display in the display system 112 (FIG. 1). Forexample, if the user accidentally selects the wrong icon when typing arespective word, the probabilistic model may be updated to account forsuch errors in the future. In an exemplary embodiment, a mistake may bedetermined based on a user activation of an icon corresponding to thedelete function. This adaptability of the portable electronic device 100(FIG. 1) may allow correction of user interface errors (such as parallaxand/or left-right symmetry) associated with which finger(s) the user isusing and how the user is holding the portable electronic device 100(FIG. 1) while using it. This functionality is discussed further belowwith reference to FIG. 14.

In some embodiments the language data structure system 1050 may includefewer or more components. Two or more components may be combined and anorder of two or more components may be changed.

Attention is now directed towards additional embodiments of userinterfaces and associated processes that may be implemented on thedevice 100 (FIG. 1). FIG. 11A is a flow diagram of an embodiment of asymbol entry process 1100. While the symbol entry process 1100 describedbelow includes a number of operations that appear to occur in a specificorder, it should be apparent that the process 1100 can include more orfewer operations, which can be executed serially or in parallel (e.g.,using parallel processors or a multi-threading environment), an order oftwo or more operations may be changed and/or two or more operations maybe combined into a single operation.

A plurality of icons may be displayed on a touch-sensitive display(1110). A respective icon may correspond to two or more symbols. Acontact by a user with the display that corresponds to selection of therespective icon may be detected (1112). A symbol in the two or moresymbols for which the contact further corresponds may be determined(1114).

FIG. 11B is a flow diagram of an embodiment of a symbol entry process1130. While the symbol entry process 1130 described below includes anumber of operations that appear to occur in a specific order, it shouldbe apparent that the process 1130 can include more or fewer operations,which can be executed serially or in parallel (e.g., using parallelprocessors or a multi-threading environment), an order of two or moreoperations may be changed and/or two or more operations may be combinedinto a single operation.

A plurality of icons may be displayed on a touch-sensitive display(1132). A respective icon may correspond to two or more symbols. A firstsymbol may belong to a first subset of symbols and a second symbol maybelong to a second subset of symbols. The first symbol may have aprobability of occurrence greater than the second symbol. A contact by auser with the display that corresponds to selection of the respectiveicon may be detected (1134). A symbol in the two or more symbols forwhich the contact further corresponds may be determined (1136).

FIG. 11C is a flow diagram of an embodiment of a symbol entry process1150. While the symbol entry process 1150 described below includes anumber of operations that appear to occur in a specific order, it shouldbe apparent that the process 1150 can include more or fewer operations,which can be executed serially or in parallel (e.g., using parallelprocessors or a multi-threading environment), an order of two or moreoperations may be changed and/or two or more operations may be combinedinto a single operation.

A plurality of icons may be displayed on a touch-sensitive display(1152). A respective icon may correspond to two or more symbols. A firstsymbol may belong to a first subset of symbols and a second symbol maybelong to a second subset of symbols. The second symbol may have aprobability of occurrence immediately following the first symbol that isless than a pre-determined value. A contact by a user with the displaythat corresponds to selection of the respective icon may be detected(1154). A symbol in the two or more symbols for which the contactfurther corresponds may be determined (1156).

FIGS. 12A-12G are schematic diagrams illustrating embodiments of a userinterface for a portable electronic device 1200. These embodiments mayutilize the symbol entry processes 1100 (FIG. 11A), 1130 (FIG. 11B)and/or 1150 (FIG. 11C) described previously. As shown in FIG. 12A, thedevice 1200 may include a keyboard 1210 with a plurality of icons. Arespective icon may include two or more symbols. A first symbol for arespective icon may be selected by the user using a first gesture. Asecond symbol for a respective icon may be selected by the user using asecond gesture. The first gesture may include a continuous contact withthe display 208 and the second gesture may include a discontinuouscontact with the display 208.

The continuous contact may include a swipe and/or a rolling motion ofthe contact. The discontinuous contact may include one or moreconsecutive taps. A respective tap may include contact with the display208 for a time interval that is less than a fourth pre-determined value,such as 0.1, 0.5 or 1s. In some embodiments, two or more consecutivetaps may correspond to a second symbol if a time interval between thetwo or more consecutive taps is less than a fifth pre-determined value,such as 0.1, 0.5 or 1 s.

In some embodiments, the first symbol is in a first subset of thesymbols in the character set displayed in the keyboard 1210 and thesecond symbol is in a second subset of the symbols in the character setdisplayed in the keyboard 1210. The first subset may have a probabilityof occurrence that is greater than a sixth pre-determined value and thesecond subset may have a probability of occurrence that is less than thesixth pre-determined value. Thus, the first subset may include symbolsthat are more likely to occur, for example, in a language (using alexicography model) and/or based on a user history. The gesture used toselect the first symbol may, therefore, be easier or quicker for theuser to make. For example, the first gesture may be a tap gesture andthe second gesture may be a swipe gesture. This is illustrated in FIG.12A. The gestures needed to select corresponding symbols for arespective icon may be indicated on the icon. For example, a dot on theicon may correspond to a tap and a horizontal line on the icon maycorrespond to a dash. This ‘tap-dash’ embodiment is an example of atwo-gesture keyboard. Additional examples are discussed below.

In some embodiments, the first symbol may have a probability ofoccurrence immediately after the second symbol that is less than aseventh pre-determined value. In some embodiments, the second symbol mayhave a probability of occurrence immediately after the first symbol thatis less than a seventh pre-determined value. This arrangement orgrouping of the symbols displayed on the icons may reduce errors whenusing the keyboard 1210 because the user will be less likely to make afirst gesture for the first symbol corresponding to a respective iconand then make the second gesture for the second symbol corresponding tothe respective icon (or vice versa). Gestures for different symbols onthe respective icon may, therefore, be separated by a time interval thatis large enough to reduce a likelihood of inadvertently selecting arespective symbol using consecutive gestures for symbols correspondingto the respective icon.

FIGS. 12B-12G illustrate additional multi-gesture keyboards. For theicons in keyboards 1212, 1214, 1216, 1218, 1220 and 1222, a first symbolfor a respective icon in these keyboards may be selected with a firstgesture (for example, a single tap) and a second symbol for therespective icon may be selected using a second gesture (for example, twoconsecutive taps). The keyboard 1222 in FIG. 12G includes some iconsthat correspond to more than two symbols. These symbols may be selectedby making additional gestures, such as three consecutive taps. In someembodiments, a second or third symbol for the respective icon may beselected by the user by first contacting a meta key, such as a shiftkey, and then contacting and/or breaking contact with the respectiveicon.

While the device 1200 has been illustrated with certain components and aparticular arrangement of these components, it should be understood thatthere may be fewer or more components, two or more components may becombined, and positions of one or more components may be changed. Forexample, the keyboards 1210, 1212, 1214, 1216, 1218, 1220 and/or 1222may include fewer or additional icons. In some embodiments, a differentcharacter set and/or different groups of symbols may be used on theicons in the keyboard 1210, 1212, 1214, 1216, 1218, 1220 and/or 1222.

In some embodiments, the user selects symbols by breaking a contact withone or more icons on the display 208. In other embodiments, however, theuser may select one or more symbols without breaking contact with thedisplay 208. For example, the user may pause or maintain contact overthe respective icon for a time interval longer than an eighthpre-determined value (such as 0.1, 0.5 or 1 s) before moving on to thenext icon and corresponding symbol. In the process, the user maymaintain contact with the display. In other embodiments, selection ofthe respective icon and corresponding symbol may occur by increasing acontact pressure with the display 208 while maintaining the contact withthe display.

A flow chart for a symbol entry process 1300 corresponding toembodiments where contact is not broken is shown in FIG. 13. While thesymbol entry process 1300 described below includes a number ofoperations that appear to occur in a specific order, it should beapparent that the process 1300 can include more or fewer operations,which can be executed serially or in parallel (e.g., using parallelprocessors or a multi-threading environment), an order of two or moreoperations may be changed and/or two or more operations may be combinedinto a single operation.

A plurality of icons may be displayed on a touch-sensitive display(1310). A respective icon may correspond to at least one symbol. Acontact by a user with the display may be detected (1312). Positions ofthe contact corresponding to a sequence of icons may be determined(1314). The at least one symbol may be selected when a respectiveposition of the contact corresponds to the respective icon for a timeinterval exceeding a pre-determined value (1316).

As discussed previously, the user may make errors when using a touchscreen in the display system 112 (FIG. 1). The device 100 (FIG. 1) may,therefore, adapt an offset between an estimated contact and an actualcontact in accordance with such errors. Feedback may be provided by theuser activating an icon corresponding to a delete key. The offset may beapplied to one or more icons. In some embodiments, there may be morethan one offset and a respective offset may be applied to a respectivesubset that includes one or more icons in a plurality of the icons in akeyboard or other user interface. The adaptation may occur continuously,after a pre-determined time interval and/or if an excessive number ofuser errors occur (e.g., as evidenced by a frequency of use of thedelete icon). The adaptation may occur during a normal mode of operationof the device 100 (FIG. 1), rather than requiring the user to implementa separate keyboard training/adaptation mode.

A flow chart for a symbol entry process 1400 corresponding to suchembodiments is shown in FIG. 14. While the symbol entry process 1400described below includes a number of operations that appear to occur ina specific order, it should be apparent that the process 1400 caninclude more or fewer operations, which can be executed serially or inparallel (e.g., using parallel processors or a multi-threadingenvironment), an order of two or more operations may be changed and/ortwo or more operations may be combined into a single operation.

A plurality of icons may be displayed on a touch-sensitive display(1410). A respective icon may correspond to at least one symbol. Acontact by a user with the display may be detected (1412). An estimatedcontact that corresponds to the respective icon and the at least onesymbol may be determined in accordance with the actual contact andpre-determined offset (1414). One or more corrections for one or moreerrors in one or more selected symbols may be received (1416). Theoffset for at least the respective icon may be modified in accordancewith the one or more received corrections (1418).

FIG. 15 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device 1500. The device 1500includes a keyboard 1510 with a plurality of icons. Different spacings(“guard bands”) are used between the icons. The guard bands betweenicons visually encourage a user to touch the center of an adjacent icon,although user contact in the guard band region may also activate thenearest icon to the contact. In some embodiments, icons near the centerof the display 208 may have a smaller guard band between adjacent iconsthan icons near an edge of the display. This may reduce errors whenusing the display 208 if it is easier for a user to select or contact arespective icon near the center of the display 208. In some embodiments,the guard band near the edge of the display 208 may be larger than thatnear the center of the display 208. Conversely, in some embodiments(opposite to what is shown in FIG. 15), icons near the center of thedisplay 208 may have a larger guard band between adjacent icons thanicons near an edge of the display. This may reduce errors when using thedisplay 208 if it is easier for a user to select or contact a respectiveicon near the edge of the display 208. In some embodiments, the guardband near the edge of the display 208 may be smaller than that near thecenter of the display 208. In some embodiments, icons near the center ofthe display 208 may be larger than icons near the edge of the display208. In some embodiments, icons at the edge of the display are abouthalf the size of the other icons because it is easier to identifycontacts corresponding to edge icons.

In some embodiments, either the size of the icons or the size of theguard bands between icons could incrementally vary between the edge ofthe display and the center of the display (e.g., from small icons at theedge to large icons in the center or from small guard bands at the edgeto large guard bands in the center).

A flow chart for a symbol entry process 1600 corresponding to suchembodiments is shown in FIG. 16. While the symbol entry process 1600described below includes a number of operations that appear to occur ina specific order, it should be apparent that the process 1600 caninclude more or fewer operations, which can be executed serially or inparallel (e.g., using parallel processors or a multi-threadingenvironment), an order of two or more operations may be changed and/ortwo or more operations may be combined into a single operation.

A plurality of icons may be displayed on a touch-sensitive display(1610). The plurality of icons may be arranged in rows in a firstdimension of the display. A first guard band in the first dimensionbetween adjacent icons in a first subset of the icons may be greaterthan a pre-determined value and a second guard band in the firstdimension between adjacent icons in a second subset of the icons may beless than a pre-determined value. A contact by the user with the displaythat corresponds to selection of the respective icon may be detected(1612). A symbol corresponding to the respective icon may be displayed(1614).

FIG. 17 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device 1700. The device 1700includes a keyboard 1710 that has a plurality of icons. A respectiveicon corresponds to two or more symbols. Some symbols may be selected bycontacting two or more icons simultaneously. A respective symbol that isselected may be displayed in the display tray 214. For example, a letter‘e’ may be selected by contacting and breaking contact with the firsticon in the first row. A letter ‘l’ may be selected by contacting andbreaking contact with the first and the second icons in the first row.The icons include visual information indicating the combinations ofcontacts with icons (also referred to as chords) that correspond togiven symbols. Keyboard 1710 is sometimes referred to as a hop-scotchkeyboard.

A flow chart for a symbol entry process 1800 corresponding to suchembodiments is shown in FIG. 18. While the symbol entry process 1800described below includes a number of operations that appear to occur ina specific order, it should be apparent that the process 1800 caninclude more or fewer operations, which can be executed serially or inparallel (e.g., using parallel processors or a multi-threadingenvironment), an order of two or more operations may be changed and/ortwo or more operations may be combined into a single operation.

A plurality of icons may be displayed on a touch-sensitive display(1810). A first icon and a second icon each correspond to two or moresymbols. A contact by a user with the display that corresponds to thefirst icon and the second icon is detected (1812). A respective symbolin the two or more symbols to which the contact corresponds may bedetermined (1814). A visual indicator corresponding to the respectivesymbol is displayed (1816).

FIG. 19 is a schematic diagram illustrating an embodiment of a userinterface for a portable electronic device 1900. A keyboard 1910 doesnot include fixed icons. Instead symbols are displayed. A nearest groupof symbols, such as three letters in a region 1912, are selected inaccordance with a user contact with the display 208. In otherembodiments, the region 1912 may include two or more symbols orcharacters. A correct set of symbols may be determined using alexicography model or system, such as that shown in FIG. 10A, inaccordance with a sequence of groups of symbols that correspond to asequence of contacts by the user. As more contacts occur, a tree ofpossible words or sets of symbols corresponding to the groups of symbolsthat have been selected may be pruned until a correct or highestlikelihood word or set of symbols is determined.

In other embodiments not shown, a respective user may play a game thatis used to determine a smallest acceptable key size for a userinterface, such as a keyboard. The smallest key size may be inaccordance with a user's manual dexterity, age, health, finger size andvision. Errors made in using the icons in a keyboard during the game mayhelp determine a minimum icon size for the respective user.

In some embodiments, icons in the embodiments of the user interfaces,such as the keyboards described above, may have an effective contactarea or a strike area that is larger than the displayed icon size. Inother embodiments, the effective contact area or strike area may belarger than the displayed icon size in at least one dimension of thedisplay 208 surface.

The foregoing descriptions of specific embodiments of the presentinvention are presented for purposes of illustration and description.They are not intended to be exhaustive or to limit the invention to theprecise forms disclosed. Rather, it should be appreciated that manymodifications and variations are possible in view of the aboveteachings. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical applications,to thereby enable others skilled in the art to best utilize theinvention and various embodiments with various modifications as aresuited to the particular use contemplated.

1. A method, comprising: at a portable electronic device with atouch-sensitive display: displaying a plurality of icons on thetouch-sensitive display, wherein a respective icon in at least a subsetof the plurality of icons corresponds to two or more symbols, a firstsymbol in the two or more symbols belongs to a first subset of symbolsand a second symbol in the two or more symbols belongs to a secondsubset of symbols, and wherein the first symbol has a probability ofoccurrence that is greater than a first pre-determined value and thesecond symbol has a probability of occurrence that is less than thefirst pre-determined value; detecting a contact by a user with thetouch-sensitive display that corresponds to a selection of therespective icon, wherein the contact includes a respective gesture; anddetermining a respective symbol in the two or more symbols for therespective icon to which the contact further corresponds.
 2. The methodof claim 1, wherein the probability of occurrence is in accordance witha user history.
 3. The method of claim 1, wherein the probability ofoccurrence is in accordance with lexicography model.
 4. The method ofclaim 3, wherein the lexicography model includes a frequency of usage ofsymbols in a language.
 5. The method of claim 1, wherein the secondsymbol has a probability of occurrence immediately following the firstsymbol that is less than a second pre-determined value.
 6. The method ofclaim 1, wherein the first symbol is selected using one or more tapgestures and the second symbol is selected using a swipe gesture, andwherein a respective tap includes making contact with thetouch-sensitive display for a time interval less than a secondpre-determined value.
 7. The method of claim 6, wherein two or moreconsecutive taps correspond to the second symbol if a time intervalbetween two or more corresponding contacts is less than a secondpre-determined value.
 8. The method of claim 1, further comprisingmodifying the displayed respective icon to indicate that the contactcorresponds to the respective symbol.
 9. The method of claim 1, furthercomprising providing a visual indicator corresponding to the respectivesymbol.
 10. The method of claim 1, wherein the first subset of symbolsincludes e, t, a, o, i, n, s, r and h.
 11. The method of claim 1,wherein the first subset of symbols includes q, e, u, I, o, a, d, g, j,l, z, c, b, n and m.
 12. The method of claim 1, wherein the first subsetof symbols includes q, c, e, h, I, l, n, o, r, t, u, w and y.
 13. Themethod of claim 1, wherein the second subset of symbols includes w, yand j.
 14. The method of claim 1, wherein the second subset of symbolsincludes w, y, p, g and j.
 15. The method of claim 1, wherein the secondsubset of symbols includes w, r, t, y, p, s, f, h, k, x and v.
 16. Themethod of claim 1, wherein the second subset of symbols includes j, v, xand z.
 17. The method of claim 1, wherein the second subset of symbolsincludes b, d, f, g, j, k, m, p, q, s, v, x and z.
 18. A computerreadable storage medium storing one or more programs, the one or moreprograms comprising instructions, which when executed by a portableelectronic device with a touch-sensitive display, cause the portableelectronic device to perform a method comprising: displaying a pluralityof icons on the touch-sensitive display, wherein a respective icon in atleast a subset of the plurality of icons corresponds to two or moresymbols, a first symbol in the two or more symbols belongs to a firstsubset of symbols and a second symbol in the two or more symbols belongsto a second subset of symbols, and wherein the first symbol has aprobability of occurrence that is greater than a first pre-determinedvalue and the second symbol has a probability of occurrence that is lessthan the first pre-determined value; detecting a contact by a user withthe touch-sensitive display that corresponds to a selection of therespective icon, wherein the contact includes a respective gesture; anddetermining a respective symbol in the two or more symbols for therespective icon to which the contact further corresponds.
 19. Agraphical user interface on a portable electronic device with atouch-sensitive display, a memory, and one or more processors to executeone or more programs stored in the memory, the graphical user interfacecomprising a plurality of icons displayed on the touch-sensitivedisplay, wherein a respective icon in at least a subset of the pluralityof icons includes two or more symbols, a first symbol in the two or moresymbols belongs to a first subset of symbols and a second symbol in thetwo or more symbols belongs to a second subset of symbols, the firstsymbol has a probability of occurrence that is greater than a firstpre-determined value and the second symbol has a probability ofoccurrence that is less than the first pre-determined value, and whereina contact by a user with touch-sensitive display that corresponds to therespective icon and includes a respective gesture is used to select arespective symbol.
 20. A portable electronic device, comprising: atouch-sensitive display; one or more processors; memory; and one or moreprograms, wherein the one or more programs are stored in the memory andconfigured to be executed by the one or more processors, the one or moreprograms including: instructions for displaying a plurality of icons onthe touch-sensitive display, wherein a respective icon in at least asubset of the plurality of icons corresponds to two or more symbols, afirst symbol in the two or more symbols belongs to a first subset ofsymbols and a second symbol in the two or more symbols belongs to asecond subset of symbols, and wherein the first symbol has a probabilityof occurrence that is greater than a first pre-determined value and thesecond symbol has a probability of occurrence that is less than thefirst pre-determined value; instructions for detecting a contact by auser with the touch-sensitive display that corresponds to a selection ofthe respective icon, wherein the contact includes a respective gesture;and instructions for determining a respective symbol in the two or moresymbols for the respective icon to which the contact furthercorresponds.